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Abstract 

This study explores middle school student learning of Next Generation Science Standards 
aligned content and practices associated with use of an innovative virtual learning environment. 
The learning environment, a computer-based game called Mission HydroSci (MHS), was 
developed with the aim of supporting student learning of water systems science along with the 
practice of scientific argumentation. Teachers implemented MHS within their science 
classrooms as a replacement unit for water science over a period of two weeks. Pre- and post- 
test data were collected from 633 students across ten schools. Data included an assessment of 
water systems content and a test of argumentation competencies. Students demonstrated 
positive and statistically significant changes in water systems understandings and 
argumentation; however, the effect sizes were modest. These results have informed several 
revisions to the MHS learning environment including modifications to the argumentation platform 
within the game and additional attention to key water systems concepts. 


Introduction 

When technology is paired with education, it has the potential to revolutionize how learning 
looks in the classroom. In this study, we study middle school students’ learning of NGSS- 
aligned content and practices through the innovative use of technology via 3-D virtual learning 
environments (VLE). Specifically, we will discuss how our team addressed a current problem in 
science education: What does a multi-dimensional, Next Generation Science Standards (NGSS) 
aligned, technology-based classroom look like? 


Virtual Learning Environments (VLE) is a common term that is used to describe 
educational software. A VLE is a web-based communications platform, that allows students, 
without limitation of time and place, to access different learning tools, such as program 
information, course content, teacher assistance, discussion boards, document sharing systems, 
and learning resources (Martins and Kellermanns, 2004). However, not all scholars agree on 
the specific elements that constitute VLE. Dillenbourg, Schnieder & Synteta (2002) have 
reviewed numerous VLEs, and argue that there are seven features of VLEs, including the ability 
to enrich physical classroom activities, the integration of technologies and multiple pedagogical 
approaches, immersion in a 3D world, and ability to co-construct the environment. Other 
researchers have specifically delineated a 3D-VLE from other VLEs, defined as an environment 
that leverages aspects of human perception by extending visual information into three spatial 
dimensions (Dalgarno & Lee, 2010). Researchers have noted (i.e. Dalgorni & Lee, 2010; 
Jacobson, Kim, Lee, Lim, & Low, 2008) that there are several studies from the mid 1990s to 
late 2000s suggesting that immersive 3D-VLEs, when designed and used appropriately, provide 
a higher level of learning and engagement when compared to more traditional 2-D learning 
techniques used to deliver equivalent educational content. While there are certainly costs 
associated with the development and implementation of 3D-VLEs, the research suggests that 
these can offer an increased level of student engagement and achievement. 


Given their potential to engage learners in an educational concept while also effectively 
at promoting learning, VLEs have been used in a wide range of educational contexts. Within 
science education, VLEs have been previously used in a diverse range of science topics. 
Additionally within the science education literature, VLEs have been used to see promote 
learning of both science content knowledge as well as scientific argumentation. 


Virtual learning environment (VLE) based interventions pertaining to science have been 
also shown to produce increases in student’s content knowledge in a a wide range of sciences, 
from physics to earth sciences. Within physics, gains were seen in seventh grader,s knowledge 
of forces after interacting with the VLE, Carrot Land. Within Carrot Land students play as a 
rabbit whose goal is to collect as many carrots as possible using the least amount of energy. 
The VLE makes the content of frictional forces accessible by limiting the variables students 
have to interact with (i.e. amount of force used and direction of force vectors). By focusing on 
these variables students are able to see how they interact without the noise of other real world 
interactions, something only possible within a VLE. As such, Carrot Land was shown to produce 
greater gains in student knowledge of frictional forces as well as forces needed to overcome 
friction (Chen, 2015) when comparing control classrooms versus classrooms who did not have 
the Carrot land intervention. 

The VLE named Selene: A Lunar Construction GaME allowed players to create and 
modify aspects of celestial bodies that were designed to have specific properties. These 
properties include water state, atmosphere composition, mass, etc. The game pushed players 
to use their content knowledge to progress through the various levels, by forcing them to apply 
knowledge of the various properties of celestial objects. As such, Selene was able to 
significantly increase student knowledge about various properties of celestial at a consistent 
rate among most users (Reese, Tabachnick, & Kosko 2015). 

VLEs have also been utilized to increase student knowledge of water systems. Tiaga 
Park, a VLE, has students involved in a mystery pertaining to sick animals in a zoo. This 
immersive simulation was shown to increase students knowledge around water systems as their 
ability to understand complex systems with multiple input and output variables. (Barab, Zuiker, 
Warren, Hickey, Ingram-Goble, Kwon, & Herring, 2007). 


Argumentation Gains through VLEs 

VLE based argumentation interventions can successfully increase student 
argumentation skills. There are many subsets of argumentation including dialogic 
argumentation, generation of arguments, evaluative knowledge and argument quality 
assessment. VLEs have been shown to increase all of these. lordanou (2015) and Kuhn (2016) 
used VLE interventions to increase the dialogic (debate) argumentation skills of students; the 
VLEs explicitly informed students into what makes a good dialogic argument and students’ 
overall skills improved versus control groups that did not participate in the VLE. 
Hafter (2014) was able to significantly affect student’s evaluative knowledge and generative 
argumentation subsets, versus a control group, using a VLE as a simple training environment 
centered around argument structure and basing arguments off of legitimate information. 


Lastly, Squire and Jan (2007) were able to produce gains in argumentation generation 
with students using the VLE, Mad City Mystery. Within Mad City Mystery, students generated 
hypotheses using science content to explain the cause of death of an in-game character. The 
researchers also found that participants’ grade levels affected gains in different aspects of 
argumentation. The middle school students improved their ability to generate hypotheses, while 
high school students showed the greatest gains in ability to evaluate the quality of arguments 
more critical of the hypothesis of their teammates. Overall VLEs have been shown to elicit gains 
in player’s dialogic argumentation, generation of arguments, evaluative knowledge and 
argument quality assessment skill sets. Based on these studies showing student gains in both 
content knowledge and argumentation competency, we have developed Mission HydroSci 
(MHS), an argumentation-based VLE focused on understanding water systems, which 
leverages the affordances of a VLE while simultaneously staying consistent with the vision of 
science and learning described within the NGSS. 


Developing a Next Generation Science Standards aligned VLE 

The Next Generation Science Standards: For States, By States (NGSS) (NRC, 2014) 
were developed to better integrate science content and scientific practices within the classroom. 
The NGSS provided a shift in focus toward essential skills to the practice of science as opposed 
to a the singular focus on content knowledge characterizing previous learning standards . To 
reach this goal, the NGSS takes the stance that scientific practices, including argumentation, 
and science content are inexorably connected, and should be reflected that way within the 
classroom. As such the NGSS describes many scientific practices but leverages the ideas from 
A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas 
(NRC, 2012, p.45), when describing argumentation as a central classroom practice. Ultimately, 
if we expect to truly reflect science within the classroom, consistent with the vision of the NGSS, 
argumentation must become a central focus. However this will require novel approaches to 
curriculum and instruction. 

In order to support the type of learning consistent with the vision of the NGSS, students 
need rich learning environments with a focus on science practices. VLEs can be leveraged to 
create these rich contexts, whereas predominant approaches used in K-12 science courses 
often fail to provide the kinds of rich contexts necessary to meet the demands of the NGSS 
(Barko & Sadler, 2013). While there have been several 3D-VLEs designed and implemented 
within science education, all of these focus on a singular dimension of science learning: either 
content or practice. As a results, the VLEs are not able to provide students’ with an experience 
that allows for development of both science content knowledge and ability to engage in science 
practice in accordance with the NGSS. 

To this end, we describe the development of a 3D-VLE called Mission HydroSci (MHS). 
This VLE helped students build more sophisticated understands of water systems while 
engaging students in the practice of argumentation. We chose to focus on argumentation due to 
its importance on the process of science (Duschl & Osbourne, 2002). At its core, science is a 
practice of discourse and argumentation about methods, goals, and explanations of natural 
phenomena imbedded in matters of history, philosophy and sociology (Duschl, 2007). As such, 
the ability to generate an explanation of a natural phenomenon that is convincing while 
simultaneously coordinating evidence and theory is vital to any scientific process (Sampson & 


Clark, 2008). To truly engage students in the process of science, argumentation must be 
foregrounded within the classroom. “Argumentation and critique are at the very center of 
science... helping to connect the ‘hands-on’ work of scientific inquiry with the ‘minds-on’ work of 
developing scientific ideas and theories” (Osborne, Henderson, MacPherson, et al., 2016, p. 
822). Due to practice of argumentation having such a central role in science, curriculum must 
reflect this practice (Duschl, Shweingruber, & Shouse, 2007). 

Overview of MHS 

This is a study of the intervention MHS, which is a VLE that is currently in development 
with the focus of providing a context for middle school students to learn water systems content, 
while also engaging in scientific argumentation. MHS is an innovative learning technology based 
curriculum which has the potential to engage students in scientific argumentation, scaffold their 
learning to argue scientifically and integrate the practices of argumentation with disciplinary core 
ideas related to water systems. MHS will be a replacement unit for water systems curriculum in 
middle school classrooms, with the full MHS unit spanning approximately two weeks of 
playing/learning time. Learning experiences within MHS will be facilitated through interactions 
within the MHS virtual environment in which students have the opportunity to explore and 
experiment with water systems, collect evidence relative to challenging water related problems, 
interacting with non player characters, and use collected evidence as the basis for building and 
critiquing arguments. Within the MHS curriculum, students progress through a series of six 
levels. Each level has been designed around a challenging problem or task that features 
progressively more complex ideas about water systems as well as more challenging 
argumentation competencies. 

The MHS curriculum has been specially designed to align with the NGSS. Central to the 
vision of NGSS is the necessary linkage of disciplinary core ideas, crosscutting concepts, and 
scientific practices. As such, the MHS curriculum allows students to engage in science 
practices, while also building a more sophisticated understanding of water systems. MHS aims 
to integrate argumentation into the core design of the curriculum, while also recognizing the 
cognitive demands that argumentation places on students. It has been shown that 
argumentation can be a difficult practice for students to engage in (Driver, Newton & Osborne, 
2000). There are many barriers to student mastery of scientific argumentation. Students tend to 
make claims that are unjustified as well as struggle with recognizing opposing arguments 
(Sadler, 2004). Sadler (2004) also found that students do not commonly use scientific evidence 
to inform arguments, and analyzing and evaluating arguments. Furthermore, Driver and 
colleagues (2000) found that (1) students struggle with construction of arguments, and (2) 
readily dismiss scientific evidence that contradicts their initial viewpoint. However, it has also 
been found that these initial barriers to mastery can be alleviated using a variety of pedagogical 
methods. Cavagnetto (2010) identified pedagogical practices that increase student 
argumentation mastery: these include immersion practice (Clark and Sampson (2007, 2008), 
creating cognitive conflict (Clark, D'Angelo, and Menekse, 2009), and explicit instruction of 
argument structure, (Venville, 2010). As such, even though there are substantial initial barriers 
to student argumentation mastery, they can be overcome using sound pedagogy. Studies 
focusing on incorporating immersion practice, explicit argumentation, and cognitive conflict 
instruction into a single intervention, especially through the means of a VLE, are lacking. As 
such, MHS intends to provide explicit instruction of argumentation structure within an immersive 


environment as a means to overcome the innate difficulties of students' engagement of scientific 
argumentation. 

Throughout the MHS curriculum, students are constantly using their understandings of 
water systems to collect data from the game environment, which is later used by students to 
build and critique arguments. This approach allows learners to engage in this practice, while 
also developing a deeper understanding of water systems content knowledge. As such, the 
focus of this study is to examine the extent to which an NGSS aligned VLE such as MHS can 
support student learning of water systems content and scientific argumentation. We aim to 
investigate the following research question: (1) To what extent can a NGSS aligned virtual 
learning environment support student learning of water systems content and scientific 
argumentation? 


Methods 


Description of MHS 

MHS is a two week instructional unit that engages students into a three dimensional VLE 
that allows students to build more sophisticated understandings of water systems while also 
engaging in science practice. From this perspective, MHS was designed to closely align with the 
NGSS since it covers both science content (DCIs) as well as science practices (i.e. scientific 
argumentation). Allowing students to engage in scientific argumentation was a key component 
to the MHS curriculum and through completing MHS students regularly argue with non-player 
characters about water systems ideas. The MHS curriculum is comprised of six units, and 
within each unit students engage in narrative problem solving that requires them to explore the 
environment to arrive at a solution. Each unit contains an argumentation scenario where the 
student must collect data from the environment, analyse that data, and use it as supporting 
evidence for their argument non-player characters within the game. For example, in the Surface 
Water unit of MHS, students are asked by another character in the game to find and deliver 
supply crates to their location. To solve this problem, students must use the river systems to 
float the supply crates back to the character’s location. In the game, each unit requires students 
to collect and analyze data from the environment in order to formulate an argument that 
addresses the problem. 

As a means to facilitate argumentation MHS includes the claimer system, which is an 
interface that had been built specifically to allow students to create scientific arguments. The 
claimer system is a drag and drop system that allows students to build arguments by dragging 
selected argument components (i.e. claim, evidence, reasoning) into the center of the screen 
where the argument is formed (Toulmin, 1958). All of the evidence that is populated into the 
claimer systems is collected by students in earlier parts of the game. When students add 
components to their argument they can see their argument being built in real time. Another key 
component of the claimer system its ability to provide immediate feedback to players. For all 
possible incorrect permutations of claim evidence, and reasoning, students are presented with 
immediate feedback that explains how to improve their argument. 

As students progress through MHS, the argumentation tasks become increasingly more 
complex and sophisticated. The design of the argumentation scenarios of MHS was influenced 


by the learning progression of scientific argumentation proposed by Osborne et al. (2016), 
which leverages cognitive load theory to accounts for the cognitive demands that argumentation 
places on students. Early units of MHS focused on specific argument components such as 
claim, evidence, and reasoning, while later unit include argumentation scenarios that require 
students to create a counter arguments to those posited by non-player characters within the 
game. 
Sample 

This study collected data from 633 middle school students across 10 schools in the 
Midwestern United States. Students completed pre/post testing surrounding a 10-day MHS 
intervention. The pre/post-test included both a water systems content assessment and an 
argumentation assessment. Both of these assessments were multiple-choice and were 
administered electronically via the online testing portal Qualtrics before and after students 
completed the MHS curriculum. 


Instrumentation 
Water Systems Assessment 

The water systems content assessment contained 24 multiple-choice questions. the 
content of the assessment as a whole can be broken down into four areas: (1) groundwater, (2) 
surface water, (3) watersheds, and (4) water cycle. Questions pertaining to these broad areas 
include a variety of formats from recalling of factual information about water systems to 
interpreting a diagram of and applying water systems ideas to answer the items successfully . 
For example, with surface water there are questions that present students with a watershed 
map including rivers and streams and asks students to predict how a dissolved pollutant would 
spread through the watershed. In addition to these questions, there are questions that ask 
students more factual based questions related to the processes of evaporation or condensation. 


Argumentation Assessment 

The argumentation assessment is a scenario-based assessment that is grounded in the 
context of evaporation. Students are presented with a scenario where a student leaves home 
for a long summer vacation and leaves a bowl of water on the porch. Upon returning, the 
student notices that the water is gone. Based on the observation that the water is gone, the 
assessment engages students with a variety of arguments about what could have happened to 
the water. As such, the entire assessment is situated within the context of evaporation, and as 
student progressed through the assessment they worked through arguments related to the 
phenomenon of evaporation and explaining what happens to water once it evaporates. The 
assessment contains 12 multiple-choice questions that address the constructs in Sampson & 
Clark (2008), and also includes items that fall at different levels of the Osborne,Henderson, 
MacPherson, Szu, Wild, & Yao (2016) learning progression for argumentation. These include 
students’ ability to identify critical components of an argument, align evidence to a given claim, 
and engage in critique of arguments. Within the assessment, there are four items relating to 
argument structure, four items that require students to align evidence to a claim, and four items 
that measure ability to engage in critique of an argument. 


Data Analysis 


We analyzed the water systems and argumentation assessments in terms of omnibus 
gains as well as gains along individual subscales. Subscales of interest on the water systems 
assessment were: (1) groundwater, (2) water cycle, (3) surface water, and (4) watersheds. 
Subscales of interest on the argumentation assessment were: (1) argument alignment, (2) 
argument structure, and (3) criticism of an argument. Statistical significance was evaluated 
using paired samples t-tests under the null hypothesis of no gain (alpha level = 0.05). Hedges 
G, an unbiased estimate of the standardized mean difference, was used to assess practical 
significance of the change in students’ test score over time on a scale of standard deviations. 


Results 

Students made significant gains in both water systems understanding (Gain = 0.99, 
SDgain = 3.11, T = 8.03, G = 0.32) and argumentation (Gain = 0.24, SDgain = 2.25, T = 2.67, G 
= 0.11) (Table 1). The practical significance of the gain in content understanding meets the 
What Works Clearinghouse (WWC) (2014) criterion for substantive importance (G >= 0.25). 
Among the subscales for water systems understanding, students made significant gains along 
all four subscales, but only knowledge of surface water characteristics (Gain = 0.37, SDgain = 
1.27, T = 7.62, G = 0.30) met the WWC criterion for a substantive change of 0.25 or greater. 
Among the argumentation subscales, the ability to analyze the structure of an argument was the 
only construct to change significantly (Gain = 0.12, SDgain = 1.42, T = 2.15, G = 0.085), but the 
significance of the gain was small. 


Table 1. Gains in students’ understanding of water systems and argumentation across the MHS 
intervention. 


Construct Pre Post Gain T G 
Water Systems Omnibus 14.84(4.02 

Score 13.85(3.68)  ) 0.99(3.11) 8.03* 0.32 
Groundwater 2.84(1.16)  2.95(1.21) 0.11(1.29) 2.06* 0.08 
Water Cycle 6.01(1.77)  6.35(1.85) 0.34(1.63) 5.21* 0.21 
Surface Water 1.79(1.14)  2.18(1.20) 0.39(1.27) 7.62* 0.30 
Watershed 3.20(1.45)  3.36(1.51) 0.16(1.65) 2.48% 0.10 


Argumentation Omnibus Score 6.85(2.16)  7.09(2.23) 0.24(2.25) 2.67* 0.11 


Align 2.53(1.12) 2.61(1.12) 0.09(1.31) 1.64 0.07 


Structure 2.10(1.18)  2.22(1.23) 0.12(1.42) 2.15* 0.09 
Criticism 2.22(0.91)  2.25(0.86) 0.03(1.2) 0.67 0.03 


* Significant at 0.05 alpha level 


Discussion 

Our data indicate that MHS was successful at supporting student learning of water 
systems content and scientific argumentation. While these gains were statistically significant, 
both exhibited small effect sizes. We view the design and development of the MHS curriculum 
as an iterative process, and the version of MHS that was used in this study was an early build 
that is still in development, and our team has used the results of this study to further develop 
new features to improve the learning outcomes of the curriculum. 


MHS Improvements 

We used our data to inform a number of changes to the early version of the MHS 
curriculum.including: explicit argumentation structure instruction, increased experience in 
critiquing the arguments of others, a reduction in the complexity of the argumentation system, 
variable feedback based on student performance and explicit descriptions of each 
argumentation context. We discuss these in turn. 

While the MHS curriculum produced statistically significant gains in students 
argumentation structure ability, the effect size was very small at .09. In reflecting on the 
curriculum, we noticed no instances of explicit instruction for students, and the inclusion of this 
explicit instruction should help students better understand the components of an argument. In 
designing this explicit instruction, we chose to develop an additional feature called the Claim, 
Reasoning, Evidence identifier (CREi). CREi provides students with a complete argument that 
consists of three sentences, one the for each argument component. CREi randomly selects one 
of the sentences of the complete argument for students to identify as being the claim, evidence, 
or reasoning. An important aspect of CREi is the immediate feedback that it provides to 
students after choosing which argument piece the selected sentence entailed. We intended 
CREi to serve as a drill and practice area for students to gain experience and receive feedback 
classifying the main pieces of a complete argument. We expect that including this explicit 
instruction around argument structure, should increase the gains associated with argument 
structure ability for students that complete the MHS curriculum. 

When looking at student gains in argumentation, we noticed that critique gains were the 
lowest of any argumentation dimension. With full acknowledgement of the argumentative 
demand critique entails, we point out that there were no opportunities for students to engage in 
argument critique in the early version of MHS. As such, we developed scenarios in which 
students engage in argumentation critique with each in-game non-player character at multiple 
points within the game. These critique scenarios leverage the Osborne et al. (2016) 
argumentation learning progression by first having students recognize missing argumentation 


structural components. Later students’ critiques are centered around the proper usage of 
components (i.e. evidence or reasoning). By including these explicit critique scenarios we 
expect increases in student's overall argumentation competency, as well as their ability to 
critique alternative arguments. 

When students were playing the game we noticed that there were a lot of students trying 
every possible permutation of claims, evidence and reasoning with little time in between 
submissions. This led us to believe that the argumentation scenarios presented within the 
CLAIMER system were too cognitively demanding and that students were not actually reading 
the feedback given to them. As such, we made alterations in an attempt to decrease the 
cognitive load on students and to get students to actually read the feedback.. To address the 
cognitive load issue, we reduced the complexity of each scenario so that they could have a 
maximum of two possible claims, five pieces of evidence and four possible reasonings. The 
previous version of the game had up to three possible claims, seven pieces of evidence and 
five possible reasonings. This leads to a large reduction of possible combinations, which 
hypothetically reduces the cognitive load for the student. 

We also noticed that students were not utilizing the feedback within the CLAIMER 
system. To address this we altered two aspects of the system. First, the art team did voice over 
for all of the feedback text. So even if the students do not read the text for one reason or the 
other, it is at least read to them by a non-player character. Second, we made the feedback more 
user friendly by varying itbased on student performance. In our current system we initially give 
students generalized feedback (e.g. “Your argument doesn’t make sense. Your reasoning does 
not explain how your evidence connects to your claim.”). However if the player submits five 
incorrect arguments, we give them more specific feedback (e.g. “Your claim and evidence 
makes sense. Try using a reasoning that connects your claim with your evidence.”). Note the 
nuance in the wording, with the generalized feedback any of the parts of the argument could be 
wrong, however with the specific feedback we tell the players which pieces are good and which 
ones to change. Ultimately this gives students more directed, usable feedback, allowing them to 
build an appropriate argument. We expect this to help students utilize the in-game feedback, 
thus allowing them to improve their arguments. 

When looking back at the MHS curriculum, we noticed that the argumentation scenarios 
lacked well developed contextual background. It is well Known that having a better 
understanding of the contextual background of an argument leads to higher argumentation 
competency (Sadler, 2004). With this in mind, we created previews for each argument that 
students complete within the game. These argument previews allow students to know the 
driving question of the argument, and the competing claims before any data are collected. Our 
goal with the argument previews are to provide students with a proper explanation of the driving 
question ahead of time so that when kids are collecting data and building their argument later on 
they are reminded and aware of why they are creating the argument. In addition to the argument 
preview, we also provide students with opportunities to ask questions about key science ideas 
related to the argument topic before they argue with the intention of allowing students to have a 
clearer understanding of the underlying science concepts. 


Significance of Results 


Based on the results of this study, we recognize that the curriculum could be improved to 
include more support for student learning of scientific argumentation. We have discussed ways 
in which we have made modifications and revisions to the curriculum to better support student 
learning of science content and practice. It is important to point out that despite the small effect 
sizes between students’ pre and post assessment scores, the MHS curriculum represents a 
significant move forward for the use of 3D-VLEs in science education. 

Most of the VLEs used in science education are shorter experiences for students and 
include much less curricular content. MHS is unique in that it provides a two-week long water 
systems VLE that is on the scale of a curricular unit. Previous VLEs used in science education 
tend to be the topic of one-to-two lessons, and not intended to be a main curriculum of an entire 
unit of instruction. Additionally, the MHS curriculum is unique in that it aligns to multiple 
dimensions of science learning described by the NGSS. The decision to focus on scientific 
argumentation as the main science practice in the game was made because engaging in 
argumentation also requires student to engine several other science practices. This includes the 
analysis and interpretation of data, and also generating explanations of science phenomena. 
With this in mind, it is also important to consider the innate difficulty of creating and critiquing 
arguments. Argumentation is a complex and cognitively demanding task for students to 
complete, and statistically significant gains in argumentation competency is promising given the 
modifications that have been made to the MHS curriculum. Even though the effect size was 
small, it is still important because MHS was able to to provide learning gains for particularly 
difficult topics such as argumentation. Furthermore, the MHS curriculum was able to obtain 
these significant gains using a non-traditional game based instruction that immersed players in 
a 3D world where students use their knowledge of water systems to engage in argumentation. 


Conclusions 

This study has allowed our team to gather information about the types of learning that 
the current version of the MHS curriculum can produce. We will further implement the changes 
and modifications discussed here and conduct another field test. In addition to learning gains, 
we will also be interested in the extent to which teacher treatment group impacts student 
learning outcomes. In our current data, we found intraclass correlations near zero for the 
teacher effect, meaning that a student’s teacher has little effect on his/her likelihood of achieving 
learning gains from MHS. We will be interested in seeing if this result is replicated in additional 
field tests with the new improvements to MHS described here. Our goal for the MHS curriculum 
is to provide students with a engaging way to build more sophisticated ideas about water 
systems, and to be able to use what they know to both create and critique arguments. 


The research described herein is supported by the US Department of Education’s Institute of 
Education Sciences (R305A150364) and Investing in Innovation (i3) program (U411C140081). 
The ideas expressed are those of our project team and do not necessarily reflect the views of 
the funders. 
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